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DETAILED ACTION 

1 . This Office Action is to provide the reference ''SpreedSkimmer: A System for Interactive 
Skimming Recorded Speech'^ which was cited in the previous Office Action. 

Claims 37-63 are currently pending in this application. Claims 1-36 have been canceled. 
Claims 37-63 have been added. Claims 37, 47, 57 and 60 are independent claims. 

It is noted that Applicant states "new claims 37-64 have been added" (see Remarks, page 
13), however, a review of the Image File Wrapper (EFW) indicates that only claims 37-63 
have been submitted. 

Specification 

2. It is noted that applicant has other related appUcations (e.g., application # 09/467,23 1 
filed 12/20/1999 and application # 09/785,063 filed 02/16/2001). It is requested that any 
related application be referred to in the first sentence of the specification. Applicant is 
also requested to supply the serial numbers of any other related applications currently 
pending before the U.S Patent & Trademark Office. 
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3. 



4. 



Double Patenting 

The nonstatutory double patenting rejection is based on a judicially created . doctrine 
grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a 
patent and to prevent possible harassment by multiple assignees. See In re Goodman, 
11 F 3d 1046, 29 USPQ2d 2010 (Fed. CH. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re van Omum, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Uogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 
418 F.2d 528, 163 USPQ 644 (CCPA 1969). 

Atimely filed terminal disclaimer in compliance with 37 C.F.R. ' 1.321(b) would 
overcome an actual or provisional rejection on this ground provided the conflicting 
application or patent is shown to be commonly owned with this application. See 37 
C.F.R. • 1.78(d). 

Effective January 1, 1994, a registered attorney or agent ofrecord may sign a terminal 
disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 
CFR 3.73(b). 



Claims 37^3 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 83-104 of copending 
Application No. 09/467, 231 (see claims 83-104 as submitted in the amendment filed 
7/06/2006). Although the conflicting claims are not identical, they are not patentably 
distinct from each other because claim 37 of the instant application and claim 83 of 
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copending Application No. 09/467, 231 both claim a data processing apparatus for 
processing media content comprised of a plurality of scenes, the apparatus 
comprising, an input unit operable to input content description data including: a 
plurality of importance attributes each associated with one of the segments and having 
a value representing a degree of contextual importance of the corresponding one of the 
segments; and an output unit operable to output at least one of the segments based on 
at least one of the importance attributes. The difference between claim 37 of the 
instant application and claim 83 of copending Application No. 09/467. 231 is claim 37 
of the instant application forther claims a context attribute having value for describing 
a context of the media content and claim 83 of copending Application No. 09/467, 231 
claims the important attributes having a value representing a degree of contextual 
importance of the corresponding one of the plurality of segments. The difference 
would have been obvious to a person of ordinary skill in the art at the time the 
invention was made since the claim limitations appear to have been reworded, 
however, the scope of the invention appears to be generally the same. 

This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in feet been patented. 



Claims 37-63 are provisionally rejected on the ground of nonstatutory obviousness- 
type double patenting as being unpatentable over claims 37-49 of copendii^ 
Application No. 09/785, 063 (see claims 37-49 as submitted in the amendment filed 
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02/13/2006). Although the conflicting claims are not identical, they are not patentably 
distinct from each other because claim 37 of the instant application and claim 37 of 
copending Application No. 09/785, 063 both claim a data processing apparatus for 
processing media content comprised of a plurality of scenes, the apparatus 
comprising, an input unit operable to input content description data including: a 
context attribute having a value for describing a context of the media content, a 
plurality of importance attributes each associated with one of the segments and having 
a value representing a degree of contextual importance of the corresponding one of the 
segments: and an output unit operable to output at least one of the segments based on 
at least one of the importance attributes. Furthermore, the differences between claim 
37 of the instant application and claim 37 of copending Application No. 09/785, 063 
would have been obvious to a person of ordinary skill in the art at the time the 
invention was made, since claim 37 of the instant application represents the invention 
in broader scope. The claim limitations appear to have been reworded, however, the 
scope of the invention appears to be generally the same. 

This is a provisional obviousness-type double patenting rejection because the conflicting 
claims have not in feet been patented. 



Claim Rejections - 35 USC § 103 
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6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained thought the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sougftt to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which die subject matter pertains. Patentability shall not be negatived by the manner in vOtich the 
invention was made. 

Thisaj^ication currently names Joint inventors. In considering patentability of the claims under 35 U.S.C. 103(a), 
Oie examiner presumes that the subject nutter ofOie various claims was commonly owned at the time any inventions 
covered therein were made absent any evidence to Oie contrary. Applicant is advised ofOte obligation under 37 
CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly <nmed at the time a later 
invention was made in order for the examiner to consider the apiAicability of 35 U.S.C. 103(c) and potential 35 
U.S.C. 102(e). (I) or(g) prior.art under 35 U.S.C. 103(a). 



Claims 37, 39-45, 47, 49-54, and 55 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Smith et al. "Video Skimming and Characterization through the 
Combination of Image and Language Understanding Techniques", 17-19 June 1997, pp. 



775-781. 



As to claims 37 and 47; 

Smith teaches a data processing apparatus and method for processing media content (e.g., 
video) comprised of a phirality of scenes (e.g., video sequences are separated into 
scenes) [see the discussion beginning at section 2, p. 776], the apparatus comprising: 



• an input unit operable to input content description data (e.g., text in the video 
provides significant information as to content of a scene) including a plurality of 
segments (e.g., segments) each for describing one of the plurality of scenes of 
media content {e.g., a scene will often contain recognizable human, as well as 
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ccqftion text to describe the scene), the content description data further including 
a phirality of importance attributes each associated with one of the segments 
(e.g.. identifying the most significant words in a given scene detection of 
objects of importance) [see pp. 776-779 and see also fig 6, 780], and 

• an output unit operable to output at least one of the segments based on at least 
one of the importance attributes (e.g., extract the significant audio and video 
information and create a "skim" video which represents a very short synopsis of 
the original; see Abstract /display only the video pertaining to a segment's 
content ... The amount of content displayed should be adjustable so the user can 
view as much or as little video as needed, from extremely compact to full-length 
video: section 1.1 ami see also fig. 7, p. 781). 

Smith does not specifically teach "a value representing a degree of contextual 
importance^. However, Smith's teaching "with prioritized video fi-amesfi-om each scene, 
we now have a suitable representation for combining the image and audio skims for the 
final skim" (see section 3.3) would suggests the claimed "a value representing a degree of 
contextual importance." 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to include Smith's teaching for "with prioritized video fi-amesfi-om each scene, 
we now have a suitable representation for combining the image and audio skims for the 
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final skim'' because it would have provided the capability for identifying the most 
significant segments in a video and effectively browsing the segments in a short time 
without losing the content of the video. 

As to claims 39 and 49: 

Smith teaches the content description data includes supplemental information (e.g.. the 
amount of content displayed should be adjustable so the user can view as much or as 
little video as needed; see section 1.1). 



As to claims 40 and 50: 

Smith teaches the media content corresponds to video data and/or audio data (e.g., vie 
information as well as audio information; see Abstract & section 1.1). 

As to claims 41 and 51: 

Smith teaches each of the phirality of segments is provided with Hnkage information 
Unking to dominant data that represents the segment (see Abstract). 

As to claims 42 and 52: 

Smith teaches the dominant data is one or more of text data, image data and audio di 
(e.g.. the audio portion; section 1.1 & audio and images embedded within the video; 
section 1.2). 
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As to claims 44 and 54: 

Smith teaches the context description data is previously generated outside of the data 
processing apparatus prior the inputting (see section 3.2). 

As to claims 43 and 53: 

Smith teaches a plurality of context attribute and a plurality of importance attributes are 
associated >vith one segment (e.g.. scenes, segments, and individual frames in video 
identi^ng the most significant words in a given scene, and for image understanding, it 
entails segmentation ofvideo into scenes objects of importance face and text and 
identification ofthe structure motion of scene; section 2). 

As to daims 45 and 55: 

Smith teaches the output unit is operable to output in response to a user query regarding 
the context (e.g.. the relevant portions of video for the segments related to their query; 
section Ll)> 



Claims 38, 46, 48, and 56- 59 are regected under 35 U.S.C. 103(a) as being unpatentable 
over Smith et al. in view of B. Arons, "SpeechSkimmer: A System for Interactively 
Skimming Recorded Speedi," ACM, March 1997, pp. 3-38. 
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As to claims 38 and 48: 

Smith does not specifically teach plurality of segments are hierarchically 
described." 

Arons teaches the plurality of segments are hierarchically described in the data structure 
portion (e.g., a hierarchy of segments can be created that roughly correspond to the 
spoken equivalents of sections, subsections, paragraphs, and sentences of a written 
document; section 2.3). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or oudines. 

As to claims 46 and 56: 

Arons teaches the context description data forther includes a plurality of time attributes 
each associated with a corresponding one of the segments for determining a start time and 
one of an end time and a duration of the scene associated with the corresponding segment 
(see sections 3.2 and 4.2.9). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or outlines. 

As to daim 57; 

The rejection of claim 37 above is incorporated herein in foil. Additionally. Smith 
teaches: 

. the segment elements that are associated with the one of context attributes 
regions from video frames that contain textual information), and having a value 
representing a degree of importance of the scene associated with the one of the 
segment elements in relation to the context of the context attribute [section 2.4], 
and 

. selection means for selecting one or more ofthe segments based on an analysis of 
the importance attributes (e.g.. choosing the "significant image and word" that 
should be included inthe skim video ... importance of each scene can be 
evaluated; section 1.2 & identifying significant objects that appear in the video 
frames is one ofthe key components for video characterization; section 2. 4). 



Arons teaches: 
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• the content description data being arranged in a hierarchy (e.g., a hierarchy of 
segments can be created that roughly correspond to the spoken equivalent of 
sections; subsections, paragraphs, and sentences of a written document; section 
2.3); 

■ . a plurality of segment elements each being a child of one of said section elements 
(see section 3.2); and 

• a plurality of time attributes each associated with a corresponding one of the 
segments for determining a start time and one of an end time and duration of the 
scene associated with the corresponding segment (see sections 3.2 and 4.2.9). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to combine the feature from Arons in the system of Smith because it would 
have provided the capability for locating important or emphasized portions of a 
recording, and selecting the equivalent of paragraph or new topic boundaries, for the 
purpose of creating audio overviews or outlines. 



As to claims 58-59: 

They include the same Umitations as in claims 44-45, respectively, and are similarly 
rejected under the same. 



I 
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Indication of Allowable Subject Matter 



8. Claims 60-63 appear to be allowable over the prior art of record, subject to the 

obviousness-type double patenting rejection detailed above, and subject to a final search, 

Response to Arguments 



). Applicant's arguments filed on 03/14/2006 have been fully considered but are deemed to 
be moot in view of the new grounds of rejection necessitated by Applicant's amendments. 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 

Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 
A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SEX MONTHS from the mailing date of this final action. 

Contact information 

1 1 Any inquiry concerning this communication or earlier conraiunications from the 

examiner should be directed to Maikhanh Nguyen whose telephone number is (571) 272- 
4093. The examiner can normally be reached on Monday - Friday from 9:00am - 5:30 
pm. If attenptsto reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor. Heather Hemdon can be reached at (571) 272-4136. 

The fex phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or PuWic PAIR Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 
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Any response to this action should be mailed to: 

Commissioner for patents 
P O Box 1450 

Alexandria, VA 22313-1450 



Maikhanh Nguyen 
AU 2176 



VWaiAMBASHORE 



